Electric camera

ABSTRACT

An electric camera includes an image sensing device with a light receiving surface having N vertically arranged pixels and an arbitrary number of pixels arranged horizontally, N being equal to or more than three times the number of effective scanning lines M of a display screen of a television system, a driver to drive the image sensing device to vertically mix or cull signal charges accumulated in individual pixels of K pixels to produce, during a vertical effective scanning period of the television system, a number of lines of output signals which corresponds to 1/K the number of vertically arranged pixels N of the image sensing device, K being an integer equal to or less than an integral part of a quotient of N divided by M, and a signal processing unit having a function of generating image signals by using the output signals of the image sensing device.

CROSS REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/386,656, filed Dec. 21, 2016, which is a continuation of U.S. patentapplication Ser. No. 14/661,227 filed Mar. 18, 2015, now U.S. Pat. No.9,544,517, which is a continuation of U.S. patent application Ser. No.14/264,243, filed Apr. 29, 2014, now U.S. Pat. No. 9,100,604, which is acontinuation of U.S. application Ser. No. 13/681,495, filed Nov. 20,2012, now U.S. Pat. No. 8,736,729, which is a continuation of U.S.application Ser. No. 12/845,266, filed Jul. 28, 2010, now U.S. Pat. No.8,339,493, issued Dec. 25, 2012, which is a continuation of U.S.application Ser. No. 10/660,710, filed Sep. 12, 2003, now U.S. Pat. No.8,059,177, issued Nov. 15, 2011, and is related to U.S. application Ser.No. 10/660,711, filed Sep. 12, 2003, now U.S. Pat. No. 7,403,226, issuedJul. 22, 2008, both of which are divisional applications of U.S.application Ser. No. 09/520,836, filed Mar. 8, 2000, now U.S. Pat. No.6,765,616, issued Jul. 20, 2004, the subject matter of all the above isincorporated by reference herein.

BACKGROUND OF THE INVENTION

The present invention relates to a photography related to video cameras,camcorders, digital still cameras and others using a solid-state imagesensing device, and more particularly to an electric camera using asolid-state image sensing device with a large number of pixels.

Electric cameras using solid-state image sensors such as CCDs(charge-coupled devices) include a so-called video camera or camcorderfor taking moving images and a so-called digital still camera for takingstill images. In recent years, video cameras with a still image takingfunction and digital still cameras with a moving image taking functionhave become available.

In a video camera to photograph moving images, it is generally assumedthat the video is viewed on a display such as television monitor andthus the camera is designed to produce output signals conforming to atelevision system such as NTSC and PAL. Therefore, the effective numberof vertically arranged pixels or picture elements on the image sensingdevice used in such a camera is determined to enable television signalsto be generated. The NTSC system, for example, performs interlacedscanning on two fields, each of which has an effective scanning linenumber of about 240 lines (the number of scanning lines actuallydisplayed on the monitor which is equal to the number of scanning linesin the vertical blanking period subtracted from the total number ofscanning lines in each field). To realize this, the image sensing devicehas about 480 pixel rows as the standard effective number of verticallyarranged pixels. That is, the signals of two vertically adjoining pixelsin each field are mixed together inside or outside the image sensingdevice to generate about 240 scanning lines, and the combinations ofpixels to be cyclically mixed together are changed from one field toanother to achieve the interlaced scanning.

Some image sensing devices to take moving images according to the NTSCsystem have an area of pixels for image stabilization added to the areaof effective pixel area, thus bringing the effective number ofvertically arranged pixels to about 480 or more. In this case, an areabeyond 480th pixels is read out at high speed during the verticalblanking period and therefore the signals thus read out are not used aseffective signals. Therefore, the video signals can only be generatedfrom those signals coming from the area of about 480 vertically arrangedpixels. When such a camera is used to photograph a still image, it isrelatively easy to generate a static image signal conforming to, forexample, JPEG (Joint Photographic Expert Group) from the signals comingfrom the same pixel area that is used to take a moving image. A problemremains, however, that the number of vertically arranged pixels obtainedis limited to around 480, making it impossible to produce more detailedstatic image signals.

In a camera having an image sensing device with the area of pixels forimage stabilization mentioned above, a method of alleviating thisproblem may involve using the entire area of effective pixels includingthe area of image stabilization pixels in photographing a still image.Even when photographing a still image, however, the photographed imageneeds to be monitored for check and, for that purpose, it is necessaryto generate signals conforming to the television system from signalsread out from all effective pixels.

An example of such a conventional camera has been proposed inJP-A-11-187306. In the camera disclosed in this publication, signalsfrom all the effective pixels are read out taking two or more times thefield period of the television system, stored in a memory means such asa field memory, and then subjected to interpolation processing fortransformation into signals conforming to the field cycle and horizontalscan cycle of television.

This conventional camera, however, requires a large processing circuit,such as field memory, for signal conversion. Another drawback is thatthe image sensing device readout cycle is a plurality of times the fieldcycle, degrading the dynamic resolution. Even with the use of thiscircuit, the number of pixels obtained as the static image signals islimited to the number of effective pixels used for moving videos plusthe area of image stabilization pixels.

In a digital still camera designed for taking still images, there hasbeen a trend in recent years toward an increasing number of pixels usedon the moving video image sensing device in order to obtain higherresolution static image signals. When taking a moving image ormonitoring the video, it is necessary to generate signals that conformto the television system. The number of pixels on such an image sensingdevice, however, does not necessarily match the number of scanning linesof the television system and therefore some form of conversion means isrequired.

The conversion means may involve, as in the video camera with the areaof image stabilization pixels, reading out signals from the imagesensing device taking a longer time than the field cycle andinterpolating them to generate television signals. This method has, inaddition to the problem described above, a drawback that the readoutcycle increases as the number of pixels increases, further degrading thedynamic resolution.

To mitigate this problem, JP-A-9-270959 discloses an apparatus whichmixes together or culls the pixel signals inside the image sensingdevice to reduce the number of signals to be read and therefore the readcycle. Although this apparatus alleviates the problem of the degradeddynamic resolution, it requires a large processing circuit such as fieldmemory to perform time-axis transformation to generate signalsconforming to the television system and the image sensing device itselfneeds to have a special structure for performing desired mixing andculling.

SUMMARY OF THE INVENTION

The present invention relates to a photography of video cameras,camcorders, digital still cameras and others using a solid-state imagesensing device, and more particularly to an electric camera using asolid-state image sensing device with a large number of pixels.

The conventional electric cameras, as described above, have drawbacksthat when taking a still picture by using a video camera, the number ofpixels is not sufficient and that when taking a moving image with astill camera, the associated circuit inevitably increases and thedynamic image quality deteriorates. Taking both moving and static imagesof satisfactory quality with a single camera is difficult to achieve. Inaddition to solving the above problems, to obtain good dynamic picturequality by using an image sensing device having a large number of pixelsintended for still images requires extracting a pixel area that is usedto realize an image stabilizing function. The conventional art andcameras do not offer a means to accomplish this function.

An object of the present invention is to provide an electric camera thatsolves these problems and which uses an image sensing device with asufficient number of pixels for still images and enables taking ofhighly detailed still images and a moving video taking with reducedimage quality degradation without increasing circuitry such as fieldmemory. It is also an object of the present invention to provide anelectric camera that can also realize the image stabilizing function.

According to one aspect of this invention, the electric camera torealize the above objectives has: an image sensing device with a lightreceiving surface having N vertically arranged pixels and an arbitrarynumber of pixels arranged horizontally, N being equal to or more thanthree times the number of effective scanning lines M of a display screenof a television system; a driver to drive the image sensing device tovertically mix or cull signal charges accumulated in individual pixelsof every K pixels to produce a number of lines of output signals whichcorresponds to the number of effective scanning lines M, K being atleast one of integers equal to or less than an integral part of aquotient of N divided by M (a number of lines of output signalscorresponds to 1/K the number of vertically arranged pixels N of theimage sensing device); and a signal processing unit to generate imagesignals by using the output signals of the image sensing device.

As explained above, since this invention eliminates the limit on thenumber of vertically arranged pixels, an electric camera can be providedwhich enables taking of highly detailed still images and a satisfactorymoving video taking by using an image sensing device with a large enoughpixel number even for still images.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing the configuration of a firstembodiment of an electric camera according to the present invention.

FIG. 2 is a schematic diagram showing the structure of an image sensingdevice in the first embodiment of the electric camera of the invention.

FIG. 3 is a drive pulse timing diagram in the first embodiment of theelectric camera of the invention.

FIG. 4 is a schematic diagram showing a mixing operation in the firstembodiment of the electric camera of the invention.

FIG. 5 is a schematic diagram showing a readout area in the firstembodiment of the electric camera of the invention.

FIG. 6 is a schematic diagram showing a mixing operation in the firstembodiment of the electric camera of the invention.

FIG. 7 is a block diagram showing the configuration of a secondembodiment of an electric camera according to the present invention.

FIG. 8 is a schematic diagram showing a mixing operation in the secondembodiment of the electric camera of the invention.

FIG. 9 is a schematic diagram showing a readout area in the secondembodiment of the electric camera of the invention.

FIG. 10 is a schematic diagram showing the structure of an image sensingdevice in a third embodiment of the electric camera according to thepresent invention.

FIG. 11 is a drive pulse timing diagram in the third embodiment of theelectric camera of the invention.

FIG. 12 is a schematic diagram showing an interpolation operation in thethird embodiment of the electric camera of the invention.

FIGS. 13A and 13B are schematic diagrams showing the arrangement ofcolor filters in the image sensing device in a fourth embodiment of theelectric camera according to the present invention.

DESCRIPTION OF THE EMBODIMENTS

Now embodiments of the present invention will be described by referringto the accompanying drawings. FIG. 1 is a block diagram showing theconfiguration of one embodiment of an electric camera according to theinvention.

In FIG. 1, reference number 1 represents a lens, 2 an aperture, 3 animage sensing device, 4 a drive circuit, 5 a gain adjust circuit, 6 ananalog-digital (A/D) conversion circuit, 7 a signal processing circuit,8 a vertical interpolation circuit to perform interpolation in avertical direction, 9 a horizontal interpolation circuit to performinterpolation in a horizontal direction, 10 a recording unit includingrecording media such as magnetic tape, semiconductor memory and opticaldisk to record a video signal, 11 a control circuit to control theseconstitutional elements according to the operating state, 12 an encodercircuit to modulate the video signal into a standard television signal,13 a digital-analog (D/A) conversion circuit, 14 a mode selector switchto change over the operation mode between the moving video taking andthe still image taking, 15 a record button to start or stop therecording, 16 a and 16 b gyro sensors to detect verticalimage-unstability and lateral image-unstability, respectively, and 17 animage-unstability decision circuit to determine the image-instabilityfrom signals output from the gyro sensors.

In the above configuration, light coming from the lens 1 through theaperture 2 is focused on a light receiving surface of the image sensingdevice 3 where it is converted into an electric signal. In thisembodiment the image sensing device 3 is of a CCD type. FIG. 2 shows thestructure of this image sensing device 3. In FIG. 2, denoted 30 arepixels each formed of a photodiode, which are arranged horizontally andvertically in a grid pattern. On these grid-arrayed pixels three typesof color filters that pass yellow (Ye), green (G) and cyan (Cy),respectively, are arranged in such a way that the combination of thesethree colors is repeated horizontally every three pixels and that thefilters of the same colors are lined vertically in so-called verticalstripes. Although an arbitrary number of pixels may be used, thisembodiment has an array of 1200 pixels vertically and 1600 pixelshorizontally. A vertical transfer unit 32 is a CCD which is driven bythree phase pulses V1, V2, V3. This CCD has a three-gate structure inwhich each pixel corresponds to three phase pulses and thus canvertically transfer a signal charge of each pixel independently.Transfer gates 31 for transferring the charge of each pixel to thevertical transfer unit 32 are commonly connected to a gate of thevertical transfer unit 32 that corresponds to the V2 pulse. An operationto transfer the charge from each pixel to the vertical transfer unit 32in response to a peak value of the pulse applied to the commonlyconnected gate and an operation to transfer the charge vertically areperformed separately. A horizontal transfer unit 33 horizontallytransfers the charges supplied from the vertical transfer units 32 andoutputs them successively through an output amplifier 34 from the outputterminal.

Referring back to FIG. 1, the operation performed when the moving videomode is selected by the mode selector switch 14 will be explained. Thenumber of vertically arranged pixels on the image sensing device in thisembodiment is 1200, so if the number of effective scanning lines in thefield of the NTSC system is assumed to be 240 lines, then verticallymixing five pixels (=1200 pixel rows/240 scanning lines) can match thenumber of lines of output signals from the image sensing device to thenumber of effective scanning lines.

However, in this embodiment, to realize the image stabilizing functiondescribed later, four vertically arranged pixels are mixed togetherduring motion image taking mode. When four vertically arranged pixelsare to be cyclically mixed together, the signals from the area of 960pixels (=240 scanning lines×4 pixels) out of the 1200 verticallyarranged pixels are used as effective signals and the remaining 240pixels (=1200 (all pixels)−960 (effective pixels)) are not used forimage forming. FIG. 3 shows the timing of a vertical drive pulse for theimage sensing device in this operation mode, with V1, V2 and V3representing three phase drive pulses applied to each gate of the CCD orvertical transfer unit 32.

In FIG. 3, in a period T1 included in the vertical blanking period, thedrive pulse V2 is held high to transfer the signal charge accumulated ineach pixel to under the V2 gate of the vertical CCD. Next, in a periodT2, while the V2 pulse is still at middle level, the V3 pulse is raisedfrom low level to middle level; next, while the V3 pulse is at middlelevel, the V2 pulse is changed from middle level to low level, afterwhich the V1 pulse is changed from low level to middle level; next,while the V1 pulse is at middle level, the V3 pulse is changed frommiddle level to low level, after which the V2 pulse is changed to middlelevel and finally the V1 pulse is changed from middle level to lowlevel. With this sequence of pulse operations, the signal charges underthe V2 gate for one pixel row are transferred and held again under theV2 gate.

By repeating this series of operations, the signal charges for a desirednumber of pixel rows can be transferred. In FIG. 3, during a period T3included in the vertical blanking period before the vertical effectivescanning period (the vertical scanning period minus the verticalblanking period which corresponds to the actually displayed image) andduring a period T4 included in the vertical blanking period after thevertical effective scanning period, the above transfer operation for onepixel row is repeated a total of 240 times to transfer the signalcharges of the 240 pixel rows not used for image generation to thehorizontal transfer unit 33 during the vertical blanking period. Forexample, if this transfer operation is performed 120 times during theperiod T3 and 120 times during the period T4, the signal charges fromupper 120 pixel rows and lower 120 pixel rows on the light receivingsurface are transferred to the horizontal transfer unit 33 during theperiod T3 and period T4 within the vertical blanking period. During eachof the subsequent periods T5 and T6 in the vertical blanking period, thehorizontal transfer unit 33 is driven for a predetermined period tooutput the charges transferred to the horizontal transfer unit 33 fromthe output terminal. These charges are not used as valid signals as theyare output during the vertical blanking period.

Next, in the vertical effective scanning period of FIG. 3, the aboveone-pixel-row transfer operation is performed four times during eachhorizontal blanking period to transfer the signal charges of four pixelrows to the horizontal transfer unit 33 where they are mixed together.Then, during a horizontal effective scanning period (the horizontalscanning period minus the horizontal blanking period which correspondsto the actually displayed image), the horizontal transfer unit 33 isdriven to read out the signal charges from the horizontal transfer unitto produce an output signal conforming to the television system. If theabove operation is performed on the A field and if, on the B field, thenumber of pixel rows transferred during the period T3 is set to 122 rowsand that during the period T4 is set to 118 rows, then the combinationof four pixel rows to be cyclically mixed together shifts by two rowsbetween the two fields, thus allowing the interlaced scanning to beperformed as shown in FIG. 4. (FIG. 4 shows the light receiving surfaceof the image sensing device and its relation to the displayed screen isvertically inverted.)

Let us return to FIG. 1. The output signal from the image sensing device3 is adjusted in gain by the gain adjust circuit 5 and then converted bythe A/D conversion circuit 6 into a digital signal. The digital signalis then processed by the signal processing circuit 7 that performs colorsignal processing and luminance signal processing, such as generation ofcolor signals, gamma correction, white balance processing and outlineenhancement. The image sensing device in this embodiment has an array ofvertical stripes of yellow (Ye), green (G) and cyan (Cy) color filters,so the color signals for Ye, G and Cy are obtained as a series of colorpoints from one line of output signals at all times no matter how manypixels are vertically combined. From these color signals three primarycolor signals R, G, B can be obtained from the following calculations.R=Ye−GB=Cy−GG=G

The R, G and B signals undergoes the white balance processing and gammacorrection processing in the signal processing circuit 7 and are thenconverted into color difference signals such as R-Y, B-Y or U and V. Theluminance signals and the color difference signals are then enteredthrough the vertical interpolation circuit 8 into the horizontalinterpolation circuit 9. In this operation state the signals just passthrough the vertical interpolation circuit 8 without being processed.The horizontal interpolation circuit 9 performs interpolation on thesignals in the horizontal direction.

FIG. 5 shows the light receiving surface of the image sensing device. Asdescribed above, in the operating state of this embodiment, the signalsread out during the vertical effective scanning period correspond to anarea having 960 of the 1200 vertically arranged pixels and a horizontalwidth of 1600 pixels, as shown shaded at A in FIG. 5. If the entirelight receiving surface of the image sensing device has a 4-to-3 (widthto height) aspect ratio, the shaded area A is more laterally elongatethan this aspect ratio. Hence, if the signals of all horizontal pixelsof the light receiving surface are displayed, for example, on an NTSCstandard television monitor with the 4-to-3 aspect ratio, the imagedisplayed is compressed horizontally and looks vertically elongate,compared with the original image. It is therefore necessary to outputduring the horizontal effective scanning period only those signalscoming from a pixel area with the horizontal width conforming to theaspect ratio of the television system, as shown by a shaded area B. Whenthe television system has an 4-to-3 aspect ratio, the number of pixelsin the horizontal width of the shaded area B is 1280 (=960 (verticaleffective pixels)×4/3).

Returning back to FIG. 1, the horizontal interpolation circuit 9performs interpolation processing on the signals from the horizontal1280 pixels to expand the signals so that they can be output over theentire horizontal effective scanning period. It also performs switchingamong different clocks as required. With the above operation, an areahaving 960 pixels in height and 1280 pixels in width is demarcated fromthe light receiving surface as signals conforming to the televisionsystem. Then, the luminance signal and the color difference signal areencoded by the encoder circuit 12 into television signals, which arethen converted by the D/A conversion circuit 13 into analog signals foroutput. When the recording is specified by the record button 15, thesignals are recorded by the recording unit 10. At this time, the signalsmay be compressed in the MPEG (Moving Picture Expert Group) format.

Next, the image stabilizing operation will be explained.Image-unstability information obtained by the gyro sensors 16 a, 16 bthat detect vertical and horizontal image-unstabilities is entered intothe image-unstability decision circuit 17, which checks the receivedinformation for the amount and direction of the image-unstability andconverts them into the number of pixels in vertical and horizontaldirections on the light receiving surface of the image sensing device.Based on the converted pixel numbers, the position of an extracted area(effective pixel area) on the light receiving surface is shifted in adirection that cancels the image-unstability. This can correct theimage-unstability. The positional shifting of the extracted area isperformed as follows. The shifting in the vertical direction can be madeby changing the number of pixel rows transferred during the periods T3and T4 in FIG. 3 and the shifting in the horizontal direction made bychanging the interpolation start position in the horizontalinterpolation circuit 9.

The operation during the moving video mode has been described above.Next, the operation performed when the static image mode is selected bythe mode selector switch 14 will be explained.

In the static image mode, too, until the recording is requested by therecord button 15, the camera outputs signals compatible with thetelevision system to monitor the angle of view. Unlike the moving videophotographing, all of the effective pixels on the image sensing deviceare used in this embodiment during the still image photographing toproduce signals with as high a resolution as possible. Hence, during themonitoring the television signals need to be generated from the signalscoming from the entire pixel area.

The image sensing device of this embodiment has 1200 vertically arrangedpixels, and the number of lines of output signals from the image sensingdevice can be made to match the number of effective scanning lines ofNTSC system, which is assumed to have 240 scanning lines, by verticallymixing five pixels (=1200/240). To make the image sensing device operatein this manner, the one-pixel-row transfer operation is performed fivetimes during each horizontal blanking period in the vertical effectivescanning period shown in the pulse timing diagram of FIG. 3. With thisoperation, the signal charges of five pixel rows can be mixed by thehorizontal transfer unit 33. As for the transfer operations during theperiods T3 and T4 in the vertical blanking period, because theinterlaced scanning is carried out, only two pixel rows are transferredduring the period T3 on the B field, with no transfer operationsperformed in other vertical blanking periods (In this embodiment,1200/240=5 with no remainder produced, so no further transfer isnecessary; if, however, a remainder occurs, the remaining pixels needonly be transferred during the periods T3 and T4).

The charges mixed by the horizontal transfer unit 33 are read out bydriving the horizontal transfer unit 33 during the horizontal effectivescanning period. With the above operations, the signal charges of allpixels on the image sensing device can be read out in a mannerconforming to the television system. The output signal from the imagesensing device 3 is, as during the moving image photographing, adjustedin gain by the gain adjust circuit 5 and converted by the A/D conversioncircuit 6 into a digital signal, which is then subjected to the colorsignal processing and the luminance signal processing in the signalprocessing circuit 7 before being entered into the verticalinterpolation circuit 8. During the static image monitoring, thevertical interpolation circuit 8 performs a vertical gravity centercorrection on the received signals.

FIG. 6 shows combinations of pixels to be cyclically mixed on the Afield and the B field and also the vertical position of the gravitycenter of the mixed signals. In the interlaced scanning, scanning linesof the A field and the B field are located at the centers of adjoiningscanning lines on other field. Hence, the signal samplings in the camerasystem for the two fields must be 180 degrees out of phase in thevertical direction. In the operating state of this embodiment, however,because five pixels are mixed together, the gravity centers of theoutput signals for the A field and the B field are deviated 36 degrees(=½ pixel or 1/10 the line-to-line distance on the same field) from theideal sampling phase difference of 180 degrees, as shown in FIG. 6. Tocorrect this requires generating a signal from two adjoining outputlines by interpolation. For example, if we let an nth output line on afield be Sn and an (n+1)th output line on the same field be Sn+1, then asignal Sn′ obtained by calculating Sn′=(Sn× 9/10)+(Sn+1× 1/10) is onewhose gravity center is shifted by 1/10 output line from the gravitycenter of the nth output line toward the (n+1)th output line. The abovecalculation can also be performed on the signals for the B field tocorrect the gravity center deviation of sampling. In this embodiment,however, to equalize the effects of interpolation of the A field and theB field, the following calculations are performed to correct the nthoutput line by 1/20 line toward the (n−1) line on the A field and by1/20 line toward the (n+1)th line on the B field.Sn′=(Sn× 19/20)+(Sn−1× 1/20)  A field:Sn′=(Sn× 19/20)+(Sn+1× 1/20)  B field:

While this embodiment performs the interpolation based on thecalculation of two adjoining lines of output signals, a greater numberof lines may be used for the interpolation processing. The output signalof the vertical interpolation circuit 8 is supplied to the horizontalinterpolation circuit 9, which in this operating state does nothing butpasses the signal. Then, as in the case of the moving imagephotographing, the signal is encoded by the encoder circuit 12 into atelevision signal, which is then converted by the D/A conversion circuit13 into an analog signal for output. As described above, the televisionsignals can be generated from all of the pixel area of the image sensingdevice also during the static image mode.

Next, the operation performed when the recording is requested by therecord button 15 will be described. During the monitoring in the staticmode, the signals are mixed together inside the image sensing device toreduce the number of signals and thereby generate television signals.During recording, however, the mixing processing is not performed andall the pixel signals need to be read out independently of each other inorder to produce high resolution signals. To realize this, theone-pixel-row transfer operation is performed only once during eachhorizontal blanking period in the vertical effective scanning periodshown in the pulse timing diagram of FIG. 3. This causes only the signalcharges for one pixel row to be transferred into the horizontal transferunit 33, which is then driven to read out the signal charges for onepixel row. Repeating this operation the number of times equal to thenumber of vertically arranged pixel rows enables the signal charges ofall pixel rows to be read out independently of each other. The transferoperation is not done during the periods T3 and T4 in the verticalblanking period.

The signal charges thus read out are adjusted in gain by the gain adjustcircuit 5 and converted by the A/D conversion circuit 6 into digitalsignals, which are then subjected to the color signal processing and theluminance signal processing in the signal processing circuit 7, afterwhich the signals are supplied through the vertical interpolationcircuit 8 and the horizontal interpolation circuit 9 to the recordingunit 10 where they are recorded. At this time, no interpolationprocessing is performed by the vertical interpolation circuit 8 orhorizontal interpolation circuit 9. The recording unit 10 may compressthe signals in the JPEG (Joint Photographic Expert Group) format, forexample. Because during the static image recording, no television signalcan be generated, an image immediately before starting the recording ora single color image is output as the television signal for monitoringpurpose. With the above operation, high resolution signals obtained fromall the pixels of the image sensing device can be recorded. Although inthis embodiment the recording unit is used commonly for the moving videomode and for the static image mode, dedicated recording units may beprovided separately for these modes.

As explained above, since there is no limit on the number of verticallyarranged pixels in this embodiment, an image sensing device with a largeenough pixel number even for still images can be used to photographhighly detailed still images and satisfactory moving images.

Further, the signal mixing and the vertical signal transfer during thevertical blanking period allow the signals from the image sensing devicewith a large number of pixels to be read out in a manner that conformsto the television system. This in turn can reduce image qualitydegradation and realize the moving image photographing with an imagestabilizing function and the monitoring during still imagephotographing.

Only the output signals from that horizontal segment which virtuallycorresponds to the television system's aspect ratio with respect to thevertical segment are extracted and output over the entire horizontaleffective scanning period of the television system. This ensures thatthe output signals obtained conform to the television system's aspectratio regardless of the extracted vertical segment position.

Further, the image sensing device is driven in such a way as to shiftthe position of the pixels to be cyclically mixed together every displaycycle of the television system in order to output interlaced signals.With this arrangement, the interlaced scanning can be performed evenwhen an image sensing device with a large number of pixels is used.

Further, the output signals produced by the mixing are interpolated sothat the gravity centers of the output signals interlaced every displaycycle have a phase difference of 180 degrees in the vertical direction.This ensures that the interlaced output signals have no deviation fromthe ideal 180-degree phase difference during interlacing even when aninterlace phase deviation would normally occur, as when odd numberedpixels are mixed together.

In this embodiment, the image sensing device has 1200 verticallyarranged pixels, and four pixels are mixed together during the movingvideo mode and five pixels during the static image mode. Because thearea of image-stabilization pixels may or may not be used and set to anydesire size, the number of pixels to be cyclically mixed together ineach mode needs only to be equal to or less than the integral part of aquotient of the number of vertically arranged pixels divided by thenumber of television system's effective scanning lines (in the aboveexample, 5 or less). (The number of vertically arranged pixels does notneed to be divisible and, in the above example, may be more than 1200).

The number of vertically arranged pixels for static image photographingneeds only to be three or more times the number of effective scanninglines on each field of the television system. In this embodiment thevertically adjoining pixels are mixed together to reduce the number ofoutput lines from the image sensing device during the vertical effectivescanning period. The number of lines of output signals can also bereduced by a so-called culling operation, by which only one line ofsignal charges of pixels is read out for every predetermined number oflines.

While in this embodiment the vertical transfer unit of the image sensingdevice is formed as a CCD that is driven by three phase pulses for eachpixel, the image sensing device may have any desired structure as longas it can realize the mixing or culling of pixels that meets the aboveconditions.

Although this embodiment described the case of NTSC system, theinvention can also be applied to other television systems, such as PALstandard, with different numbers of effective scanning lines.

In summary, a variety of constructions essentially equal in the workingprinciple to this embodiment can be realized by the use of an imagesensing device that has an arbitrary number of vertically arrangedpixels N three or more times the number of effective scanning lines M ofeach field of the television system and which allows the vertical mixingor culling of that number of pixels which is at least one of integersequal to or less than the integral part of a quotient of N divided by M(a number of lines of output signals corresponds to 1/K the number ofvertically arranged pixels N of the image sensing device).

Next, another embodiment of the present invention will be described byreferring to FIG. 7 showing the configuration of the embodiment. Theconfiguration shown in FIG. 7 differs from that of FIG. 1 in that it hasa view angle change switch 18. In FIG. 7 constitutional elementsidentical with those shown in FIG. 1 are assigned like reference numbersand explanations on the constitutional elements performing the sameoperations as those in FIG. 1 are omitted here.

The operations of the moving video mode, the monitoring during thestatic image mode and the static image recording are similar in normalcondition to the operations of the previous embodiment which wasexplained with reference to the configuration diagram of FIG. 1. Anoperation performed when during the moving video mode a request tochange the angle of view is made by the view angle change switch 18 willbe described.

In the normal condition of this embodiment, as explained in the previousembodiment, the mixing of four vertically arranged pixels, the verticaltransfer during the vertical blanking period and the horizontalinterpolation processing are performed to extract an area of 960 pixelsin height and 1280 pixels in width from the entire pixel area togenerate television signals. When a view angle change (which means azooming function without image quality degradations in the verticaldirection) is requested by the view angle change switch 18, threevertically arranged pixels are mixed together and the signals from theexcess vertically arranged pixels are read out during the verticalblanking periods before and after the vertical effective scanningperiod.

In this embodiment, signals of 480 pixels (=1200−240×3) or 160 lines ofoutput signals after mixing (=480/3) need to be read out during thevertical blanking period. This allows the signals of 720 verticallyarranged pixels to be read out as 240 lines of output signals conformingto the television system. To carry out this reading requires, in thepulse timing diagram of FIG. 3, transferring the signals of three pixelrows during each horizontal blanking period in the vertical effectivescanning period and also transferring a total of 480 pixel rows (=160lines of output signals) during the T3 and T4 periods in the verticalblanking period. The combinations of pixels to be cyclically mixedtogether are changed from one field to another to achieve the interlacedscanning.

The output signals of the image sensing device 3 are supplied to thegain adjust circuit 5. Because the signal level produced as a result ofthe 3-pixel mixing is ¾ the signal level of the 4-pixel mixing, theagain of the gain adjust circuit 5 is increased to 4/3 the gain of the4-pixel mixing to make the 3- and 4-pixel-mixed input signal levels tothe subsequent circuit equal. Then, the signals are processed by the A/Dconversion circuit 6 and the signal processing circuit 7 before beingsupplied to the vertical interpolation circuit 8. The combinations ofpixels to be cyclically mixed on the A field and the B field and thevertical positions of the gravity centers of the mixed signals are shownin FIG. 8. As in the static image monitoring of the previous embodiment,the phase difference between the two fields is 180 degrees. Because thesampling phases of the fields are deviated from the ideal 180-degreephase difference, the vertical position of the gravity centers arecorrected by the vertical interpolation circuit 8. The amount of phasedeviation in this operating state is 60 degrees (=½ pixel or ⅙ theline-to-line distance on the same field). To correct the phase deviationevenly on the both fields, the following calculations should beperformed.Sn′=(Sn× 11/12)+(Sn−1× 1/12)  A field:Sn′=(Sn× 11/12)+(Sn+1× 1/12)  B field:

As described earlier, the interpolation processing may use three or moreoutput lines. Next, the horizontal interpolation circuit 9 horizontallyexpands the signals from that horizontal segment which corresponds tothe 4-to-3 aspect ratio with respect to the 720 vertically arrangedpixels (i.e., signals from a horizontal 960-pixel segment (=1600×720/1200) in this operating state) so that the expanded signals can beoutput during the entire horizontal effective scanning period. With theabove operation, an area of 720 pixels in height and 960 pixels in widthcan be extracted from the light receiving surface.

Next, when the view angle change is requested again by the view anglechange switch 18, two vertically arranged pixels are mixed together, anarea of 480 vertically arranged pixels is read out during the verticaleffective scanning period, and the horizontal interpolation circuit 9expands the signals from the horizontal 640-pixel segment and outputsthe expanded signals during the entire horizontal effective scanningperiod to extract an area of 480 pixels in height and 640 pixels inwidth. (During the two-pixel mixing, because the interlace phasedeviation does not occur, the gravity center position correction by thevertical interpolation circuit 8 is not performed.) If a further viewangle change is requested by the view angle change switch 18, theoperation is restored to a normal state where four vertically arrangedpixels are mixed together.

As a result of the above operation, the area extracted from the lightreceiving surface of the image sensing device can be changed to threedifferent areas: (A) 960 pixels high by 1280 pixels wide, (B) 720 pixelshigh by 960 pixels wide and (C) 4880 pixels high by 640 pixels wide.That is, the angle of view can be changed to three different angles.With the area A produced by the 4-pixel mixing taken as a reference, thearea B produced by the 3-pixel mixing can provide an image enlarged by1.33 times and the area C produced by the 2-pixel mixing can provide animage enlarged by two times. It should be noted here that because thethree different areas are chosen by changing the number of pixel to becyclically mixed together in order to make the number of the effectiveoutput lines from the imaging device agree with the number of theeffective scanning lines of the television system, the angle of view canbe changed while maintaining a good image with no image qualitydegradation in the vertical direction, when compared with an ordinaryso-called digital zoom which generates effective scanning lines ofsignals by interpolating a small number of output lines. During themonitoring of a static image, too, it is possible to perform the similaroperation of changing the angle of view by changing the number of pixelrows to be cyclically mixed together.

As described above, in addition to the advantages provided by theprevious embodiment, this embodiment can also realize the view anglechange with little image quality degradation even for still images byusing an image sensing device with a large number of pixels and changingthe number of pixels to be cyclically mixed together.

Further, because changes in signal level caused when the number ofpixels to be cyclically mixed is changed are absorbed by the gain adjustmeans, the input signal level to the subsequent signal processing meanscan be kept constant.

While in this embodiment, the view angle change is performed by the viewangle change switch 18, the angle of view may be changed continuously bya zoom switch. In this case, when the magnification factor does notreach the value that is obtained by changing the pixel mixing, thedigital zoom performs the ordinary interpolation processing. In thisembodiment, when the magnification factor is 1 or more and less than1.33, the 4-pixel mixing is performed; for the factor of 1.33 or moreand less than 2, the 3-pixel mixing is done; and for the factor of 2 orhigher, the 2-pixel mixing is carried out. The mixing operation may beinterlocked with an optical zooming mechanism.

Regardless of the number of pixels in the image sensing device, thestructure of the image sensing device or the television system employed,this embodiment, as in the previous embodiment, may also use an imagesensing device that has an arbitrary number of vertically arrangedpixels N three or more times the number of effective scanning lines M ofeach field and which allows the vertical mixing or culling of thosenumbers of pixels which are at least two of integers equal to or lessthan the integral part of a quotient of N divided by M. The use of thisimage sensing device can form a variety of constructions essentiallyequal in the working principle to this embodiment.

Next, a further embodiment of the present invention will be described.The overall configuration of this embodiment is similar to that of FIG.1, except that the inner structure of the image sensing device 3 isdifferent. The configuration of the image sensing device in thisembodiment is shown in FIG. 10. In FIG. 10, denoted 30 are pixels formedof photodiodes, which are arranged horizontally and vertically in a gridpattern. On these grid-arrayed pixels three types of color filters thatpass yellow (Ye), green (G) and cyan (Cy), respectively, are arranged inso-called vertical stripes.

In this embodiment, the image sensing device has an array of pixelsmeasuring 864 pixels vertically and 1152 pixels horizontally. A verticaltransfer unit 32 is a CCD which is driven by six phase pulses V1, V2,V3, V4, V5, V6. This CCD has a two-gate structure in which each pixelcorresponds to two phase pulses and six gates corresponding to the sixphase pulses are repeated for every three pixels. Transfer gates 31 fortransferring the signal charge of each pixel to the vertical transferunit 32 are commonly connected to respective gates of the verticaltransfer unit 32 corresponding to the pulses V1, V3, V5. An operation totransfer the signal charge from each pixel to the vertical transfer unit32 in response to peak values of pulses applied to the commonlyconnected gates and an operation to transfer the charge vertically areperformed separately.

A horizontal transfer unit 33 horizontally transfers the chargessupplied from the vertical transfer units 32 and outputs themsuccessively through an output amplifier 34 from the output terminal.This image sensing device, unlike the one in the previous embodiment,cannot vertically transfer all pixels independently of each other, butcan mix together the signal charges of three vertically adjoining pixelsinside the vertical transfer unit 32 before transferring them.

First, the operation performed in this embodiment during the movingvideo mode will be explained. In the image sensing device of thisembodiment the effective number of vertically arranged pixels is 864. Ifthree pixels are vertically mixed, the signals of the 720 (=240×3) ofthe 864 vertically arranged pixels can be used as the effective signalsand the remaining 144 (=864−720) pixels can be used as theimage-unstability correction pixel area.

FIG. 11 shows the timings of vertical drive pulses for the image sensingdevice of FIG. 10 during this operation mode, with V1, V2, V3, V4, V5,V6 representing the six phase drive pulses applied to the respectivegates of the CCD or vertical transfer unit 32. In FIG. 11, during aperiod T1 included in the vertical blanking period, the drive pulses V1,V3 and V5 are held high to cause the signal charge of each pixel to betransferred to under the V1, V3 and V5 gates of the vertical CCD. Then,the V2 and V4 pulses are changed from low level to middle level to mixthe charges of the adjoining three pixels. After the mixing, the V5pulse is changed from middle level to low level to hold the mixed signalcharges under the V1, V2, V3, V4 gates.

Next, a series of operations performed during a period T2 (changing thedrive pulses from middle level to low level or from low level to middlelevel in the order of V1, V2, V3, V4, V5 and V6) causes one mixed outputline (3 pixel rows) to be transferred and held again under the V1, V2,V3, V4 gate. By repeating this series of operations, a desired number ofoutput lines of mixed signal charges can be transferred.

In FIG. 11, during a period T3 included in the vertical blanking periodbefore the vertical effective scanning period and during a period T4included in the vertical blanking period after the vertical effectivescanning period, the transfer operation for one output line is repeateda total of 144 times to transfer 144 output lines of signal charges notused for image forming to the horizontal transfer unit 33 at high speedduring the vertical blanking periods. During subsequent periods T5 andT6 in the vertical blanking periods, the horizontal transfer unit 33 isdriven for predetermined periods to output the signal chargestransferred to the horizontal transfer unit 33 from the output terminal.

Next, in the vertical effective scanning period of FIG. 11, theone-output-signal-line transfer operation is performed during eachhorizontal blanking period. Then, during the horizontal effectivescanning period, the horizontal transfer unit 33 is driven to read outthe signal charges from the horizontal transfer unit 33. With thisoperation the signal charges of three pixels mixed together can be readout in a way conforming to the television system. As shown in FIG. 11,the signals for the A field are mixed by changing the V2 and V4 pulsesto middle level after transferring the signals from the pixels to thevertical transfer unit 32. The signals for the B field, on the otherhand, are mixed by changing the V2 and V6 pulses to middle level. Withthis mixing method, the combinations of pixels to be cyclically mixedtogether can be changed from one field to another, thereby realizing theinterlaced scanning. The output signals from the image sensing deviceare processed in the similar manner to that of the previous embodiment.A vertical interpolation circuit 8 performs the gravity centercorrection, as in the 3-pixel mixing in the previous embodiment, and ahorizontal interpolation circuit 9 performs interpolation processing tomatch the aspect ratio with that of the television system.

Next, the operation during the monitoring in the static image mode willbe explained. It is assumed that the still image photographing is doneby using all effective pixels of the image sensing device, as in theprevious embodiment. The image sensing device of this embodiment has 864vertically arranged pixels and, when 3-pixel mixing is done as in themoving video taking, the number of output lines is 288 (=864/3), whichmeans that these signal lines cannot be read out in a manner conformingto the television system. Hence, during the monitoring in the staticimage mode, vertical 6-pixel mixing is performed. The 6-pixel mixing canbe achieved by transferring to the horizontal transfer unit 33 in eachhorizontal blanking period two output lines of signal charges each ofwhich line has been generated by vertically mixing three pixels withinthe vertical transfer unit 32. The 6-pixel mixing can reduce the numberof output lines from the image sensing device down to 144 (=864/6)lines. The output signals of the image sensing device that were reducedto 144 output lines are interpolated by the vertical interpolationcircuit 8 to transform the 144 output lines of signals into 240 lines ofsignals, which conform to the television system. To generate 240 linesof signals from the 144 lines requires interpolation processing thatgenerates five lines from three lines ( 144/240=⅗).

FIG. 12 shows how the interpolation is performed using two adjoiningoutput lines. Let three output lines of the image sensing device be n,n+1 and n+2. The five output lines of signals can be generated from thefollowing calculations.n′=nn′+1=n/2+(n+1)/2n′+2=n+1n′+3=(n+1)/2+(n+2)/2n′+4=n+2

Three or more output lines of signals may be used for interpolationprocessing. With the above operation, television signals can begenerated by using signals of all pixels of the image sensing devicealso during the monitoring in the static image mode.

Next, the operation performed when the recording is requested by therecord button 15 will be explained. In the recording process, the mixingprocessing is not performed and signals of all pixels need to be readindependently of each other in order to obtain high-resolution signals.To realize this, the aperture 2 is first closed and then, during theperiod T2 in the pulse timing diagram of FIG. 11, only the V1 pulse israised to high level to transfer the signal charge of only the pixeladjacent to the V1 gate to the vertical transfer unit 32. Then, thevertical transfer unit 32 and the horizontal transfer unit 33 aresuccessively driven to read out the signal charges. Similarly, the V3pulse is raised to high level to read the signal charge of the pixeladjacent to the V3 gate, followed by raising the V5 pulse to high levelto read the signal charge of the pixel adjacent to the V5 gate. With theabove processing, the signal charges of all pixels can be read outindependently in three successive operations. The signal charges thusread out are recorded in the recording unit 10. At this time, they arerearranged properly to reconstruct the pixel arrangement on the lightreceiving surface of the image sensing device.

As described above, this embodiment offers the following advantages. Ifthe number of vertically arranged pixels is not an integral multiple ofthe number of scanning lines of the television system, the signalsconforming to the television system can be generated from the whole areaof effective pixels by performing the pixel mixing and the verticalinterpolation.

In this embodiment, as in the previous embodiment, regardless of thenumber of pixels in the image sensing device, the structure of the imagesensing device or the television system employed, a variety ofconstructions essentially equal in the working principle to thisembodiment can be realized by using an image sensing device that has anarbitrary number of vertically arranged pixels N three or more times thenumber of effective scanning lines M of each field and which allows thevertical mixing or culling of that number of pixels which is greater byat least one than the integral part of a quotient of N divided by M.

Next, a further embodiment of the present invention will be explained.This embodiment differs from the previous embodiments in that the imagesensing device have different arrangements of color filters. FIGS. 13Aand 13B show arrangements of color filters in this embodiment. Thesecolor filters in both examples are arranged in vertical stripes and,regardless of the number of pixels to be vertically mixed or culled, theR, G, B primary color signals can be generated from one line of outputsignals. FIG. 13A shows a color filter arrangement on the image sensingdevice in which white filters (W=passing all colors) are used instead ofthe green (G) filters used in the previous embodiment. The R, G, Bsignals can be obtained by the following calculations.R=W−CyG=Ye+Cy−WB=W−Ye

When this color filter arrangement is used, a higher sensitivity can beobtained than the color filter arrangement of the previous embodiment.FIG. 13B show a color filter arrangement that uses, in stead ofcomplementary colors, color filters that pass the primary colors R, G,B. This color filter arrangement can directly produce the primary colorsignals, R, G, B and can provide a camera with good color purity andgood color S/N.

With the above color filter arrangements, it is possible to producecolor signals corresponding to the three kinds of color filters fromeach line of output signals at all times no matter how many pixels arevertically mixed or culled. Therefore, color signals conforming to thetelevision system can be generated easily.

The invention claimed is:
 1. A camera comprising: an image sensor havingan array of pixels arranged vertically and horizontally in a gridpattern; a processor to form a plurality of image signals by using thepixels of the image sensor in a static image mode and a moving videomode, the processor configured to: form the image signals by using afirst set of the pixels of the image sensor corresponding to apredetermined first view angle during recording in the static imagemode, and form the image signals by using a second set of the pixels ofthe image sensor corresponding to a predetermined second view angleduring recording in the moving video mode, wherein the predeterminedsecond view angle is different from the predetermined first view angle;an image-instability detector configured to detect an amount ofimage-instability of the camera and configured to change a position ofthe second set of the pixels according to the amount ofimage-instability detected by the image-instability detector, in orderto correct the image-instability; and a display configured to: display astill image corresponding to the image signals formed based on the firstset of the pixels, and display a moving image corresponding to the imagesignals formed based on the second set of the pixels.
 2. The cameraaccording to claim 1, wherein the position of the second set of thepixels is changed to perform instability correction caused by verticalmovement of the camera.
 3. The camera according to claim 1, wherein theposition of the second set of the pixels is changed to performinstability correction caused by horizontal movement of the camera. 4.The camera according to claim 1, wherein the position of the second setof the pixels is changed to perform instability correction caused byvertical and horizontal movement of the camera.
 5. The camera accordingto claim 1, wherein the processor is further configured to perform whitebalance processing on one or more of the first or second set of thepixels.
 6. The camera according to claim 1, wherein the processor isfurther configured to perform luminance signal processing on one or moreof the first or second set of the pixels.
 7. The camera according toclaim 1, wherein the processor is further configured to perform colorsignal processing on one or more of the first or second set of thepixels.
 8. The camera according to claim 1, further comprising: aninterpolator configured to perform interpolation operations on one ormore of the first or second set of the pixels.
 9. The camera accordingto claim 1, further comprising: an encoder configured to encode thestill image with a first encoding format and for encoding the movingimage with a second encoding format, wherein the first encoding formatis different from the second encoding format.
 10. The camera accordingto claim 1, further comprising: a memory configured to record the stillimage after the still image is encoded by a first encoding method andfor recording the moving image after the moving image is encoded by asecond encoding method that is different from the first encoding method.11. An operating method for a camera comprising: receiving light with animage sensor having an array of pixels arranged vertically andhorizontally in a grid pattern; forming a plurality of image signals byusing the pixels of the image sensor in a static image mode and a movingvideo mode, including: forming the image signals by using a first set ofthe pixels of the image sensor corresponding to a predetermined firstview angle during recording in the static image mode; and forming theimage signals by using a second set of the pixels of the image sensorcorresponding to a predetermined second view angle during recording inthe moving video mode, wherein the predetermined second view angle isdifferent from the predetermined first view angle; detecting an amountof image-instability of the camera and changing a position of the secondset of the pixels according to the amount of detected image-instability,in order to correct the image-instability; displaying a still imagecorresponding to the image signals formed based on the first set of thepixels; and displaying a moving image corresponding to the image signalsformed based on the second set of the pixels.
 12. The method accordingto claim 11, wherein the position of the second set of the pixels ischanged to perform instability correction caused by vertical movement ofthe camera.
 13. The method according to claim 11, wherein the positionof the second set of the pixels is changed to perform instabilitycorrection caused by horizontal movement of the camera.
 14. The methodaccording to claim 11, wherein the position of the second set of thepixels is changed to perform instability correction caused by verticaland horizontal movement of the camera.
 15. The method according to claim11, further comprising: performing white balance processing on one ormore of the first or second set of of the pixels.
 16. The methodaccording to claim 11, further comprising: performing luminance signalprocessing on one or more of the first or second set of the pixels. 17.The method according to claim 11, further comprising: performing colorsignal processing on one or more of the first or second set of thepixels.
 18. The method according to claim 11, further comprising:performing interpolation operations on one or more of the first orsecond set of the pixels.
 19. The method according to claim 11, furthercomprising: encoding the still image with a first encoding format andencoding the moving image with a second encoding format, wherein thefirst encoding format is different from the second encoding format. 20.The method according to claim 11, further comprising: recording thestill image after the still image is encoded by a first encoding stepand recording the moving image after the moving image is encoded by asecond encoding step that is different from the first encoding step.